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@ Graphics system. 



@ A graphics system comprises an input device, 
and a computer for detecting changes in the position 
of the Input device relative to a surface, and a 
monitor (63) for displaying patterns which follow the 
movement of the input device. The computer cal- 
culates at least one differential derivative with re- 
spect to time of the input device position and con- 
trols characteristics of the display patterns in depen- 
dance upon the or each calculated derivative. 
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The present invention relates to a graphics 

systenn. 

Computer graphics systems are known which 
convert the movements of a pen-like Input across a 
tablet into patterns on a screen, the patterns cor- 
responding in shape to the movements of the input 
device relative to the tablet. Variations of this basic 
system are also known in which for example the 
input device is pressure sensitive such that the 
width of the pattern is a function of the applied 
pressure, or the input device is in the form of a 
"mouse" which is movable on a ball across a flat 
surface, rotations of the ball controlling the configu- 
ration of the resultant pattern. 

Such systems must respond rapidly to move- 
ments of the input device if the user is to be able 
to exercise adequate control. If changes to the 
displayed image lagged significantly behind the 
movements of the input device causing those 
changes the user would lose the movement/image 
change visual feedback that is essential. 

Systems are available which provide an ade- 
quate rate of response to input device movement 
but users still feel inhibited by the system char- 
acteristics which are considered not to adequately 
emulate the expressive qualities of images pro- 
duced using traditional physical media. For exam- 
ple, the type of mark produced when painting with 
a soft brush is completely different from that pro- 
duced when using pen and ink, and the "feel" of 
the different implements is very different. The abil- 
ity of an artist to express himself or herself in the 
visual arts appears to be significantly affected by 
the behaviour of the media used. Available systems 
do not adequately emulate the dynamic behaviour 
of traditional media. 

IBM TDB, Vol. 24, No. 4. September 1981, 
pages 2013-2016, describes a digitising tablet and 
pen for creating graphics. The pen can be made to 
simulate a paint brush so that for example if the 
pen is moved slowly or fast the amount of colour 
laid down is varied. 

GB-A-21 40257 also describes a system for 
simulating a real paint brush by responding to the 
pressure with which an artist's tool is pressed 
against the underlying surface. 

It is an object of the present invention to pro- 
vide an improved computer graphics computer 
system which obviates or mitigates the problems 
outlined above. 

In accordance with the present invention, a 
graphics system comprises an input device, means 
for detecting changes in the position of the input 
device relative to a surface, means for displaying 
patterns which follow the movement of the input 
device, and storage means for storing data defining 
the form of a simulated surface on which the pat- 
terns are being drawn, wherein the means for dis- 
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playing patterns is responsive to movement of the 
input device and the stored data to simulate the 
effect of the underlying, simulated surface on the 
pattern being drawn after passage of the input 
5 device. 

This allows real effects to be simulated elec- 
tronically such as the slope of a surface on which 
the pattern is being drawn. Thus, the data stored 
by the storage means can define the effect of 
10 gravity on the coloured medium which is being 
modelled while another effect which can be mod- 
elled is the effect of the thickness of the previously 
laid down medium such as simulated ink on a 
subsequent pass of the input device. 
75 Embodiments of a computer graphics system 

according to the present invention will now be 
described, by way of example, with reference to 
the accompanying drawings, in which: 6 

Figure 1 is a representation of a three-dimen- 
20 sional scalar and vector field derivable from a 
pressure sensitive input device: 
Figure 2 is a flow chart illustrating the mode of 
operation of an embodiment of the present in- 
vention; 

25 Figure 3 is a flow chart illustrating field model- 
ling in the embodiment of Figure 2; 
Figure 4 illustrates data derivable from the em- 
bodiment of Figure 2 to serve as the basis for 
generating predetermined effects; 
30 Figure 5 illustrates effects which can be gen- 
erated using the embodiment of Figure 2; 
Figure 6 illustrates the generation of inertial ef- 
fects using the embodiment of Figure 2; 
Figure 7 illustrates three-dimensional modelling 
35 to give a three-dimensional field database; 

Figure 8 is a flow diagram illustrating one exam- 
ple of the mode of operation of the system; and. 
Figure 9 illustrates the appearance of a "drip" 
generated by the system when operated in ac- 
40 cordance with the Figure 8 flow diagram. 

Referring to Figure 1, the illustration schemati- 
cally represents the movement of a pressure sen- 
sitive input device or pen (not shown) of conven- 
tional form over a square tablet 1 , the pen following 
45 the line 2. In known manner, the position in terms 
of X and Y coordinates relative to the tablet of the 
pen tip is output from the tablet, and the pressure 
on the pen tip is output from the pen and can be 
considered as representing a third dimension Z. 
50 Thus the system provides X, Y and Z coordi- 

nates which are conventionally used to trace a line 
on a screen of a monitor (not shown) correspond- 
ing in shape to the shape of the line 2 and having a 
width related to pen tip pressure. Alternative sys- 
55 tems to the pen and tablet system described pro- 
duce two and three-dimensional Information in a 
similar manner to produce similar screen effects. 
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In a conventional systenn, the frame store de- 
fining pixels on a one to one basis with the nnonitor 
screen is continually updated with the Information 
from the tablet and thereafter controls the monitor 
display in a conventional manner. 

With the known arrangements, the screen ef- 
fects produced are intended to be independent of 
the speed at which the input device moves and its 
acceleration. If the system response time is rela- 
tively slow some speed-related effects may be- 
come apparent, simply because the system cannot 
"keep up with" the input device movement. Such 
effects are however not intended. 

At least one differential derivative with respect 
to time is calculated by the system and used to 
adjust the image resulting from movement of the 
Input device. This can be done in a number of 
ways as described below. 

Referring to Figure 2, a flow chart is shown 
illustrating the operation of one embodiment of the 
invention. Input coordinate information 50 in three 
dimensions corresponding to the movement of an 
input device such as a simulated paint brush is 
processed in accordance with a predetermined cal- 
culation procedure 51 which is controlled by a 
timing signal from a clock 52. The timing signal 
determines the period between successive evalu- 
ations of X, Y and 2 (sampling intervals) and the 
calculation procedure 51 derives the first, second 
and third derivatives with respect to time of X. Y 
and Z. 

The absolute position (X.Y.Z) 53 in 3D space 
(or the absolute position (X,Y) in 2D space together 
with a pressure value (Z)) is provided to a calcula- 
tion routine 54. as is the velocity 55 (the vector 
sum of the first derivatives), the acceleration 56 
(the vector sum of the second derivatives), and the 
rate of change of acceleration 57 (the vector sum 
of the third derivatives). The sign of the accelera- 
tion 58 and of the rate of change of acceleration 59 
are also provided. This enables the effect of the 
"brush" on the "sheet" to be modelled. This mod- 
elling may be achieved independently of any pre- 
vious patterns which have been applied to the 
sheet or can take account of the previous patterns 
stored in a three-dimensional field model 61 . If this 
model is used then the model is updated following 
each calculation 54. The use of the model will be 
explained in more detail below with reference to 
Figures 3 and 7. 

The media to be emulated, e.g. soft brush and 
paint, pen and ink etc, is modelled in a media 
behaviour database 60 which defines various fixed 
parameters such as brush shape. The field and 
media databases provide data to enable the effect 
of moving the brush to be determined and re- 
corded in a real time dynamic database 62 (the 
frame store) to generate appropriate Image data for 
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display on an output device 63 such as the screen 
of a monitor. 

The various elements shown in Figure 2 will 
normally be implemented on a suitably pro- 
5 grammed computer with attached stores although 
hard wired circuits could also be used. 

Figure 3 illustrates the operation of the calcula- 
tion routine 54 which enables the field model 61 to 
incorporate the results of previous Inputs Into a 

10 basic field model. For example, a pre-defined 
three-dimensional field model 61 can be estab- 
lished and stored corresponding to a flat sheet of 
paper inclined to the vertical at a predetermined 
angle corresponding to the position adopted by an 

75 artists easel. Information relating to the position, 
velocity etc. of the input device is submitted to the 
field transformation calculation routine 54 which 
takes into account the pre-defined three-dimen- 
sional field model 61 and the media behaviour 

20 database 60 to produce desired effects, e.g. paint 
slippage In the direction of the bottom edge of the 
simulated paper, and provides transformed vector 
and scalar outputs 64 accordingly. In the illustrated 
embodiment, the transformed outputs are fed back 

25 to the field model 61 to update the predefined 
model, thereby enabling subsequent inputs to be 
affected by earlier inputs. For example, thick hori- 
zontal layers of paint could affect the slippage of 
overlying vertical layers of paint, and this can be 

30 taken into account in the updated field model. It Is 
not necessary however for the field model to be 
updated by a feedback input in this way. 

Referring now to Figure 4, one example of a 
graphic effect which can be achieved in accor- 

35 dance with the invention is illustrated. An input 
device is caused to follow the path indicated by the 
heavy line 3. The movement is Initially straight 
from point 4 to point 5, and then there is an abrupt 
change in direction after which the input device 

40 initially follows a straight line but then follows an 
arcuate path. The lines 6 Indicate the position of 
the input device at successive timing intervals 
(each timing interval may correspond to a predeter- 
mined number of position sampling intervals). Thus 

45 the spacing between adjacent lines 6 along the 
path increases with speed and the change in the 
spacing and the relative inclinations between adja- 
cent lines 6 is a measure of acceleration. In the 
example of Figure 4, the system has been set up 

50 to "paint" circular shapes the size of which in- 
creases with velocity. This is equivalent to the sort 
of effects one can achieve with a heavily loaded 
dripping brush or fountain pen. (Depending on the 
effect desired, the width/velocity relationship could 

55 be reversed so that the resultant pattern is thinner 
as the velocity increases. The relationship does not 
have to be directly proportional). The system Is 
also set up to simulate splash effects such as 
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occurs when a heavily loaded brush is rapidly 
decelerated, e.g. at point 5. The input device was 
decelerated as it approached point 5 from point 4, 
and thus the splash effect 7 is in line with the initial 
direction of nnovement of the input device. The 
splash effect is triggered if the calculated accelera- 
tion exceeds a preset threshold determined by the 
media behaviour database 60. 

Figure 5 illustrates the results achieved with a 
system operating as described above with refer- 
ence to Figure 4. It will be seen that the splash 
effect is represented as three "drips" (as shown at 
65) positioned in a line orthogonal to the accelera- 
tion direction. 

Figure 6 illustrates how extra effects can be 
achieved. A loaded soft brush which is rapidly 
accelerated exhibits inertial effects as the shape of 
the brush changes with acceleration. The effect of 
velocity, acceleration and/or pressure on flexure of 
bristles of the brush can be modelled in the media 
database. In Figure 6, an input device is assumed 
to have moved along a straight line path from point 
8 to point 9, along a curved path from point 9 to 
point 10, and then along a straight line path from 
point 10 to point 11. Shapes 12 made up from 
circular leading sectors and triangular trailing sec- 
tions are "painted" at intervals the spacing be- 
tween which is speed dependent. The speed is 
thus roughly constant but the velocity changes. As 
the input device accelerates to move around the 
curve from point 9 to point 10, the trailing end of 
the shape 12 swings out away from the line 9, 10. 
the degree to which the shape swings out being a 
function of acceleration perpendicular to the ve- 
locity vector and possibly also the pressure applied 
to the input device. Thus the inertial effects of a 
heavily loaded soft brush can be simulated. The 
pattern displayed can be a string of shapes 12. or 
the envelope indicated by lines 13 and 14 swept 
out by the shapes 12 can be filled in. 

Another example of effects Which can be 
achieved will now be described with reference to 
Figures 8 and 9. In this example, an elliptical brush 
is simulated and the splash effect due to accelera- 
tion modelled. Initially, in a step 70 the media 
behaviour database 60 is loaded with fixed param- 
eters including proportionality constants D1 and k 
and thresholds D2 and VC. In operation, the cal- 
culation procedure 51 (Figure 2) is initiated and this 
starts by determining the current coordinates IX lY 
from the tablet over which the pen is moved and 
also determines the pen status (step 71). These 
coordinates are sampled at successive, equally 
spaced time intervals determined by the clock 52. 
The previous X coordinate (OLDIX) is obtained 
from a register in a step 72 and then the difference 
between the two X coordinates per unit time is 
determined (step 73) to obtain the X velocity 
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(ACX). Similarly, the previous Y coordinate (OLDIY) 
is obtained from a register (Step 74), and the 
difference in Y coordinates per unit time is deter- 
mined so as to define the Y velocity (ACY) in a 

5 step 75. The signs of ACX and ACY are examined 
and if these are each positive and both ACX and 
ACY exceed the threshold VC (step 76) then a 
colour index is incremented by one. The colour 
index is stored in the media database 70 and 

70 defines a series of different colours which may 
graduate in terms of brightness from a compara- 
tively dark colour to a very bright colour with in- 
creasing colour index value or can be a completely 
artificial set of colours. The effect of this procedure 

75 is that the faster the pen is moved, the brighter the 
colour becomes. 

In a step 77 it is determined whether the signs 
of the velocities ACX and ACY are both negative 
but yet both ACX and ACY exceed the threshold 

20 VC in which case the colour index is decreased by 
one. 

A velocity register is updated at each sampling 
interval with the current velocity so that in a step 
78 the previous X velocity is read from the register 

25 and the difference between the previous X velocity • 
and the current X velocity per unit time is deter- 
mined so as to define an acceleration variable 
ACACX (step 79) which is stored. In a similar 
manner the previous Y velocity is read from a 

30 register (step 80) and the difference between the 
present Y velocity and the previous Y velocity per 
unit time is determined in order to define the Y 
acceleration variable ACACY (step 81 ). 

If the pen state is non-zero (step 82) indicating 

35 that the splash effect is desired, then in a step 83 it 
is determined whether the X acceleration exceeds 
a threshold D2 and if it does the position of secon- 
dary drips in the X direction is determined. This 
distance is determined to be proportional to the X 

40 velocity, the proportionality constant being the val- 
ue k previously stored. 

In a similar manner, in a step 84, the Y accel- 
eration is compared with the threshold D2 and the 
displacement of secondary drips in the Y direction 

45 is determined. 

Following step 84, or where appropriate step 
82, a step 85 is performed in which the size of the 
main spot or primary drip from the brush is deter- 
mined. This size can vary dynamically depending 

50 on the values of the X and Y velocities and is 
defined as an elipse with major axes in the X and Y 
directions of D1 ACX and D1 ACY respectively 
(Figure 9). 

In a step 86 the coordinates of the centre of 
55 the primary and secondary drips are determined 
and the real-time dynamic database updated ac- 
cordingly so that pixels corresponding to the pri- 
mary and secondary drips are loaded with a suit- 

4 
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able time delay with data defining the drips in 
terms of colour densities with the result that the 
primary and secondary drips are then successively 
displayed (step 87) on the monitor 63. 

Finally, the various registers recording X, Y 5 
coordinates velocities and accelerations are up- 
dated (step 88) and processing returns (step 89) to 
step 71 . 

Figure 9 illustrates a typical form for a primary 
drip 90 and a set of three secondary drips 91 . The lo 
dimensions of the primary drip 90 are defined in 
Figure 9 and the displacement of the secondary 
drips 91 from the Y axis is also defined. In this 
case, it will have been found in step 83 that the 
acceleration variable in the X direction exceeds D2 75 
but that the acceleration variable in the Y direction 
does not exceed D2. Consequently, the secondary 
drips 91 are not displaced in the Y direction. 

Further effects to those described can be 
achieved, for example textures dependent upon 20 
velocity, colours which are dependent on velocity, 
and the like. The effects do not necessarily have to 
emulate effects found with traditional media. 

Claims 25 

1. A graphics system comprising an input device, 
means (1) for detecting changes in the position 
of the input device relative to a surface, and 
means (63) for displaying patterns which follow 30 
the movement of the input device, charac- 
terised by storage means (61) for storing data 

defining the form of a simulated surface on t^*:*- 
which the patterns are being drawn, wherein v- . 

the means for displaying patterns is responsive 35 ^..cL.' 
to the movement of the input device and the *• 
stored data to simulate the effect of an under- 
lying surface on the pattern being drawn, after 
passage of the Input device. 



40 



A system according to claim 1. wherein the 
stored data simulates the thickness of a simu- 
lated colour medium laid down previously. 
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(54) Graphics system 

(57) A graphics system comprises an input device, 
and a computer for detecting changes in the position of 
the input device relative to a surface, arrd a monitor (63) 
for displaying patterns which follow the movement of the 
input device. The computer calculates at least one dif- 



ferential derivative with respect to time of the input device 
position and controls characteristics of the display pat- 
terns In dependance upon the or each calculated deriv- 
ative. 
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